Haptic conversion system using segmenting and combining

ABSTRACT

A system is provided that converts an input into one or more haptic effects using segmenting and combining. The system receives an input. The system further segments the input into a plurality of input sub-signals. The system further converts the plurality of input sub-signals into a haptic signal. The system further generates the one or more haptic effects based on the haptic signal.

FIELD

One embodiment is directed generally to a device, and more particularly,to a device that produces haptic effects.

BACKGROUND

Haptics is a tactile and force feedback technology that takes advantageof a user's sense of touch by applying haptic feedback effects (i.e.,“haptic effects”), such as forces, vibrations, and motions, to the user.Devices, such as mobile devices, touchscreen devices, and personalcomputers, can be configured to generate haptic effects. In general,calls to embedded hardware capable of generating haptic effects (such asactuators) can be programmed within an operating system (“OS”) of thedevice. These calls specify which haptic effect to play. For example,when a user interacts with the device using, for example, a button,touchscreen, lever, joystick, wheel, or some other control, the OS ofthe device can send a play command through control circuitry to theembedded hardware. The embedded hardware then produces the appropriatehaptic effect.

Devices can be configured to coordinate the output of haptic effectswith the output of other content, such as audio, so that the hapticeffects are incorporated into the other content. For example, an audioeffect developer can develop audio effects that can be output by thedevice, such as machine gun fire, explosions, or car crashes. Further,other types of content, such as video effects, can be developed andsubsequently output by the device. A haptic effect developer cansubsequently author a haptic effect for the device, and the device canbe configured to output the haptic effect along with the other content.However, such a process generally requires the individual judgment ofthe haptic effect developer to author a haptic effect that correctlycompliments the audio effect, or other type of content. Apoorly-authored haptic effect that does not compliment the audio effect,or other type of content, can produce an overall dissonant effect wherethe haptic effect does not “mesh” with the audio effect or othercontent. This type of user experience is generally not desired.

SUMMARY

One embodiment is a system that converts an input into one or morehaptic effects using segmenting and combining. The system receives aninput. The system further segments the input into a plurality of inputsub-signals. The system further converts the plurality of inputsub-signals into a single haptic signal, or multiple haptic signals thatcan either be played separately on different haptic output devices, ormixed into a single haptic signal. The system further generates the oneor more haptic effects based on the haptic signal.

BRIEF DESCRIPTION OF THE DRAWINGS

Further embodiments, details, advantages, and modifications will becomeapparent from the following detailed description of the preferredembodiments, which is to be taken in conjunction with the accompanyingdrawings.

FIG. 1 illustrates a block diagram of a system in accordance with oneembodiment of the invention.

FIG. 2 illustrates a flow diagram of haptic conversion functionalityperformed by a system, according to an embodiment of the invention.

FIG. 3 illustrates a plurality of input sub-signals that are analyzedand converted into one or more haptic effects using frequency bandanalysis and prioritization, according to an embodiment of theinvention.

FIG. 4 illustrates a flow diagram of haptic conversion functionalityperformed by a system, according to another embodiment of the invention.

FIG. 5 illustrates an example of shifting a frequency of an inputsignal, according to an embodiment of the invention.

FIG. 6 illustrates a block diagram of a haptic mixer configured to mix aplurality of haptic sub-signals, according to an embodiment of theinvention.

FIG. 7 illustrates a flow diagram of the functionality of a hapticconversion module, according to an embodiment of the invention.

DETAILED DESCRIPTION

One embodiment is a system that can convert an input, such as an audiosignal, into a haptic signal that can be used to generate hapticeffects. The system can filter the input into multiple frequency bands,where each frequency band includes a sub-signal of the input, and thesystem can further prioritize the multiple frequency bands based on oneor more parameters of analysis. In general, the system can create aprioritized list of the frequency bands. The system can further selectone or more frequency bands from the prioritized list, and use theselected frequency band(s) to generate the haptic signal, where thehaptic signal is based upon, at least in part, a combination of theselected frequency band(s). For example, the system can band-pass filteran audio signal into four frequency bands, and can create a hapticsignal based on the frequency band that contains an audio sub-signalwith the highest magnitude. The haptic conversion functionality of thesystem can be implemented either as offline functionality that providesan output file that can be played back on a device, or as an algorithmthat performs the processing at playback time.

Another embodiment is a system that can convert an input, such as anaudio signal, into a haptic signal that can be used to generate hapticeffects. The system can read a multimedia file (e.g., audio file orvideo file), and extract an input signal, such as an audio signal, fromthe multimedia file. The system can filter the input signal into one ormore input sub-signals. For example, the system can use different bandpass filters with complementary cutoff frequencies to segment the inputsignal into different complementary input sub-signals. The system canthen create a haptic sub-signal for each input sub-signal. The systemcan further mix, or otherwise combine, the haptic sub-signals into anoverall haptic signal that corresponds to the original input signal. Thehaptic conversion functionality of the system can be implemented as asoftware module, a mobile application, or a plug-in for audio/videoplayers and editing tools.

FIG. 1 illustrates a block diagram of a system 10 in accordance with oneembodiment of the invention. In one embodiment, system 10 is part of amobile device, and system 10 provides a haptic conversion functionalityfor the mobile device. In another embodiment, system 10 is part of awearable device, and system 10 provides a haptic conversionfunctionality for the wearable device. Examples of wearable devicesinclude wrist bands, headbands, eyeglasses, rings, leg bands, arraysintegrated into clothing, or any other type of device that a user maywear on a body or can be held by a user. Some wearable devices can be“haptically enabled,” meaning they include mechanisms to generate hapticeffects. In another embodiment, system 10 is separate from the device(e.g., a mobile device or a wearable device), and remotely provides thehaptic conversion functionality for the device. Although shown as asingle system, the functionality of system 10 can be implemented as adistributed system. System 10 includes a bus 12 or other communicationmechanism for communicating information, and a processor 22 coupled tobus 12 for processing information. Processor 22 may be any type ofgeneral or specific purpose processor. System 10 further includes amemory 14 for storing information and instructions to be executed byprocessor 22. Memory 14 can be comprised of any combination of randomaccess memory (“RAM”), read only memory (“ROM”), static storage such asa magnetic or optical disk, or any other type of computer-readablemedium.

A computer-readable medium may be any available medium that can beaccessed by processor 22 and may include both a volatile and nonvolatilemedium, a removable and non-removable medium, a communication medium,and a storage medium. A communication medium may include computerreadable instructions, data structures, program modules or other data ina modulated data signal such as a carrier wave or other transportmechanism, and may include any other form of an information deliverymedium known in the art. A storage medium may include RAM, flash memory,ROM, erasable programmable read-only memory (“EPROM”), electricallyerasable programmable read-only memory (“EEPROM”), registers, hard disk,a removable disk, a compact disk read-only memory (“CD-ROM”), or anyother form of a storage medium known in the art.

In one embodiment, memory 14 stores software modules that providefunctionality when executed by processor 22. The modules include anoperating system 15 that provides operating system functionality forsystem 10, as well as the rest of a mobile device in one embodiment. Themodules further include a haptic conversion module 16 that converts aninput into one or more haptic effects using segmenting and combining, asdisclosed in more detail below. In certain embodiments, hapticconversion module 16 can comprise a plurality of modules, where eachmodule provides specific individual functionality for converting aninput into one or more haptic effects using segmenting and combining.System 10 will typically include one or more additional applicationmodules 18 to include additional functionality, such as Integrator™software by Immersion Corporation.

System 10, in embodiments that transmit and/or receive data from remotesources, further includes a communication device 20, such as a networkinterface card, to provide mobile wireless network communication, suchas infrared, radio, Wi-Fi, or cellular network communication. In otherembodiments, communication device 20 provides a wired networkconnection, such as an Ethernet connection or a modem.

Processor 22 is further coupled via bus 12 to a display 24, such as aLiquid Crystal Display (“LCD”), for displaying a graphicalrepresentation or user interface to a user. The display 24 may be atouch-sensitive input device, such as a touch screen, configured to sendand receive signals from processor 22, and may be a multi-touch touchscreen.

System 10, in one embodiment, further includes an actuator 26. Processor22 may transmit a haptic signal associated with a generated hapticeffect to actuator 26, which in turn outputs haptic effects such asvibrotactile haptic effects, electrostatic friction haptic effects, ordeformation haptic effects. Actuator 26 includes an actuator drivecircuit. Actuator 26 may be, for example, an electric motor, anelectro-magnetic actuator, a voice coil, a shape memory alloy, anelectro-active polymer, a solenoid, an eccentric rotating mass motor(“ERM”), a linear resonant actuator (“LRA”), a piezoelectric actuator, ahigh bandwidth actuator, an electroactive polymer (“EAP”) actuator, anelectrostatic friction display, or an ultrasonic vibration generator. Inalternate embodiments, system 10 can include one or more additionalactuators, in addition to actuator 26 (not illustrated in FIG. 1).Actuator 26 is an example of a haptic output device, where a hapticoutput device is a device configured to output haptic effects, such asvibrotactile haptic effects, electrostatic friction haptic effects, ordeformation haptic effects, in response to a drive signal. In alternateembodiments, actuator 26 can be replaced by some other type of hapticoutput device. Further, in other alternate embodiments, system 10 maynot include actuator 26, and a separate device from system 10 includesan actuator, or other haptic output device, that generates the hapticeffects, and system 10 sends generated haptic signals to that devicethrough communication device 20.

System 10, in one embodiment, further includes a speaker 28. Processor22 may transmit an audio signal to speaker 28, which in turn outputsaudio effects. Speaker 28 may be, for example, a dynamic loudspeaker, anelectrodynamic loudspeaker, a piezoelectric loudspeaker, amagnetostrictive loudspeaker, an electrostatic loudspeaker, a ribbon andplanar magnetic loudspeaker, a bending wave loudspeaker, a flat panelloudspeaker, a heil air motion transducer, a plasma arc speaker, and adigital loudspeaker. In alternate embodiments, system 10 can include oneor more additional speakers, in addition to speaker 28 (not illustratedin FIG. 1). Further, in other alternate embodiments, system 10 may notinclude speaker 28, and a separate device from system 10 includes aspeaker that outputs the audio effects, and system 10 sends audiosignals to that device through communication device 20.

System 10, in one embodiment, further includes a sensor 30. Sensor 30can be configured to detect a form of energy, or other physicalproperty, such as, but not limited to, sound, movement, acceleration,bio signals, distance, flow, force/pressure/strain/bend, humidity,linear position, orientation/inclination, radio frequency, rotaryposition, rotary velocity, manipulation of a switch, temperature,vibration, or visible light intensity. Sensor 30 can further beconfigured to convert the detected energy, or other physical property,into an electrical signal, or any signal that represents virtual sensorinformation. Sensor 30 can be any device, such as, but not limited to,an accelerometer, an electrocardiogram, an electroencephalogram, anelectromyograph, an electrooculogram, an electropalatograph, a galvanicskin response sensor, a capacitive sensor, a hall effect sensor, aninfrared sensor, an ultrasonic sensor, a pressure sensor, a fiber opticsensor, a flexion sensor (or bend sensor), a force-sensitive resistor, aload cell, a LuSense CPS² 155, a miniature pressure transducer, a piezosensor, a strain gage, a hygrometer, a linear position touch sensor, alinear potentiometer (or slider), a linear variable differentialtransformer, a compass, an inclinometer, a magnetic tag (or radiofrequency identification tag), a rotary encoder, a rotary potentiometer,a gyroscope, an on-off switch, a temperature sensor (such as athermometer, thermocouple, resistance temperature detector, thermistor,or temperature-transducing integrated circuit), microphone, photometer,altimeter, bio monitor, camera, or a light-dependent resistor. Inalternate embodiments, system 10 can include one or more additionalsensors, in addition to sensor 30 (not illustrated in FIG. 1). In someof these embodiments, sensor 30 and the one or more additional sensorsmay be part of a sensor array, or some other type of collection ofsensors. Further, in other alternate embodiments, system 10 may notinclude sensor 30, and a separate device from system 10 includes asensor that detects a form of energy, or other physical property, andconverts the detected energy, or other physical property, into anelectrical signal, or other type of signal that represents virtualsensor information. The device can then send the converted signal tosystem 10 through communication device 20.

FIG. 2 illustrates a flow diagram of haptic conversion functionalityperformed by a system, according to an embodiment of the invention. Inone embodiment, the functionality of FIG. 2, as well as thefunctionality of FIG. 4 and the functionality of FIG. 7, are eachimplemented by software stored in memory or other computer-readable ortangible media, and executed by a processor. In this embodiment, eachfunctionality may be performed by a haptic conversion module (such ashaptic conversion module 16 of FIG. 1). In other embodiments, eachfunctionality may be performed by hardware (e.g., through the use of anapplication specific integrated circuit (“ASIC”), a programmable gatearray (“PGA”), a field programmable gate array (“FPGA”), etc.), or anycombination of hardware and software.

The haptic conversion functionality can include receiving one or more“chunks” (also identified as segments) of an input signal, processingeach chunk of the input signal, and playing back the modified chunk ofthe input signal on a haptic output device, such as an actuator. Incertain embodiments, the input can be an audio signal, or other type ofaudio input, that includes audio data. In other alternate embodiments,the input can be a video signal, or other type of video input, thatincludes video data. In yet other alternate embodiments, the input canbe an acceleration signal, or other type of acceleration input, thatincludes acceleration data. In yet other alternate embodiments, theinput can be an orientation signal that includes orientation data, anambient light signal that includes ambient light data, or another typeof signal that can be related to a media file, and that can also besensed by a sensor, such as sensor 30. The output of the sensor can berecorded beforehand, and can be provided along with the media file.Thus, the sensor may be, or may not be, attached to the system.

According to the embodiment, the flow begins at 210, where an inputsignal chunk is received. As previously described, an input signal chunkis a segment of an input signal. In one embodiment, the input signalchunk can include the entire input signal. The flow proceeds to 220.

At 220, the input signal chunk is filtered (e.g., band-pass filtered) tocreate a plurality of input sub-signals (also identified as frequencybands). More specifically, one or more filters (e.g., band-pass filters)can be applied to the input signal chunk to remove segments of the inputsignal chunk, so that the remaining segment of the input signal chunkincludes one or more frequencies within a specific frequency band. Thesegment of the input signal chunk that remains after the application ofthe filter is identified as an input sub-signal, or a frequency band. Inembodiments involving a plurality of filters, each filter can correspondto a specific frequency band, the plurality of filters can be applied tothe input signal chunk in multiple passes (where a different filter isapplied to the input signal chunk in each pass), and each filter cancreate a segment of the input signal chunk that includes one or morefrequencies within a frequency band that corresponds to the filter. Incertain embodiments, an input sub-signal can include the entire inputsignal chunk.

For example, a first filter can represent a low frequency band, a secondfilter can represent a medium frequency band, and a third filter canrepresent a high frequency band. A first filter can be applied to aninput signal chunk, and a first input sub-signal can be created, wherethe first input sub-signal includes one or more frequencies within a lowfrequency band. A second filter can then be applied to the input signalchunk, and a second input sub-signal can be created, where the secondinput sub-signal includes one or more frequencies within a mediumfrequency band. A third filter can then be applied to an input signalchunk, and a third input sub-signal can be created, where the thirdinput sub-signal includes one or more frequencies within a highfrequency band.

The input sub-signals (i.e., frequency bands) that are created at 220are illustrated in FIG. 2 as frequency bands 221, 222, 223, and 224.However, any number of input sub-signals can be created, and each inputsub-signal can be defined to include one or more frequencies within anytype of frequency band. An example implementation of input sub-signalsis further described below in conjunction with FIG. 3. The flow thenproceeds to 230.

At 230, the input sub-signals (e.g., frequency bands 221-224) areprioritized based on an analysis parameter. More specifically, one ormore characteristics of each input sub-signal are analyzed, where theanalysis parameter defines the one or more characteristics. Examples ofthe characteristics can include: frequency, duration, envelope, density,and magnitude. The input sub-signals can then be ordered (i.e.,prioritized) based on the analyzed characteristics of each inputsub-signal. For example, a prioritized list of the one or more inputsub-signals can be generated, where the one or more input sub-signalsare prioritized within the list based on the analyzed characteristics ofeach input sub-signal.

As an example, an analysis parameter can be defined as a magnitude of aninput sub-signal. A plurality of input sub-signals (i.e., frequencybands) can be created, and each input sub-signal can be analyzed inorder to determine a maximum magnitude value. The maximum magnitudevalue for each input sub-signal can be compared, and the inputsub-signals can be prioritized based on the corresponding maximummagnitude values.

An example implementation of analyzing and prioritizing inputsub-signals using an analysis parameter is further described below inconjunction with FIG. 3. The flow then proceeds to 240.

At 240, a haptic signal is calculated and generated based on one or moreinput sub-signals that are selected from the prioritized inputsub-signals. More specifically, one or more input sub-signals are firstselected from the prioritized input sub-signals. For example, an inputsub-signal with the highest priority (or a plurality of inputsub-signals with the highest priorities) can be selected from aprioritized list of the input sub-signals. Subsequently, a haptic signalis calculated based on the selected input sub-signal(s). Morespecifically, the haptic signal is calculated to include one or morecharacteristics of the selected input sub-signal(s). In certainembodiments, the haptic signal can be calculated to include all thecharacteristics of the selected input sub-signal(s). In embodimentswhere there are a plurality of input sub-signals, the haptic signal canbe calculated, at least in part, based on a combination of the inputsub-signals. The haptic signal is subsequently generated.

As an example, an input sub-signal with a highest maximum magnitudevalue can be selected. The maximum magnitude value of the inputsub-signal can be used to calculate a haptic signal. This haptic signalcan subsequently be used to generate a haptic effect that is based onthe input sub-signal with the highest magnitude. As another example, thethree input sub-signals with the three highest maximum magnitude valuescan be selected. The maximum magnitude values of the three inputsub-signals can be used to calculate a haptic signal. For example, anaverage, or other calculation, of the three maximum magnitude values canbe calculated. This average value, or other calculated value, can beused to generate a haptic signal. This haptic signal can subsequently beused to generate a haptic effect that is based on the three inputsub-signals with the three highest magnitudes. The flow then proceeds to250.

At 250, the generated haptic signal is “warped” into a “warped hapticsignal.” More specifically, the generated haptic signal is input into a“warping” function, where a warping function can “warp” or transform thedata contained within an input signal to create an output signal, wherethe output signal is also identified as a “warped signal.” Thus, byinputting the generated haptic signal into the warping function, thewarping function can warp the data contained within the generated hapticsignal. The warping of the data contained within the generated hapticsignal can ultimately transform the generated haptic signal into the“warped haptic signal.” In certain embodiments, the warped haptic signalis more suitable for a specific haptic output device, and thus, thewarped haptic signal can be played on the specific haptic output devicein order to generate one or more haptic effects.

In one example, for an LRA or an ERM, the warping function can envelopean input haptic signal, or use a maximum magnitude value of the inputhaptic signal, to calculate a magnitude of an output haptic signal thatis generated, where the magnitude of the output haptic signal correlatesto the magnitude of a haptic effect generated by the output hapticsignal. In another example, for a piezoelectric actuator, a highbandwidth actuator, or an EAP actuator, the warping function can playback the input haptic signal as a waveform, or perform another type ofalgorithm to convert the input haptic signal into an output hapticsignal.

In certain embodiments, 250 can be omitted. The flow then proceeds to260.

At 260, the haptic signal (either the warped haptic signal if 250 isperformed, or the generated haptic signal if 250 is omitted) is sent toa haptic output device, such as actuator 26 of FIG. 1, and the hapticoutput device generates one or more haptic effects based on the hapticsignal. Thus, the one or more haptic effects can be generated based onthe selected input sub-signal(s). The flow then ends.

In certain embodiments, the haptic conversion functionality illustratedin FIG. 2 can be performed in real-time on a device configured to outputhaptic effects, such as a mobile device or touchscreen device. In otheralternate embodiments, the haptic conversion functionality illustratedin FIG. 2 can be performed offline, by a computer or other type ofcomputing machine, and a resulting haptic signal can be sent to thedevice that is configured to output the haptic effects.

FIG. 3 illustrates a plurality of input sub-signals that are analyzedand converted into one or more haptic effects using frequency bandanalysis and prioritization, according to an embodiment of theinvention. As previously described, an input signal 301 can be filteredto create a plurality of input sub-signals using one or more filters,where the plurality of input sub-signals includes input sub-signals 302,303, and 304. Input sub-signal 302 represents a 200 Hz center frequencyband-pass of an input signal, and can be created using a first filter.Input sub-signal 303 represents a 1000 Hz center frequency band-pass ofan input signal, and can be created using a second filter. Inputsub-signal 304 represents a 5000 Hz center frequency band-pass of aninput signal, and can be created using a third filter.

According to the embodiment, each input sub-signal can divided into aplurality of segments or windows. In the illustrated embodiment, examplewindows 310, 320, and 330 are illustrated, where windows 310, 320, and330 each include a segment of input sub-signals 302, 303, and 304. Inalternate embodiments, an input sub-signal can include other windows notillustrated in FIG. 3. Further, in certain embodiments, the windows ofan input sub-signal can be in a sequential arrangement, where asubsequent window starts at a position when a preceding window ends.

According to the embodiment, for each window, one or morecharacteristics of each input sub-signal can be analyzed based on ananalysis parameter. Thus, in the illustrated embodiment, for each windowof windows 310, 320, and 330, one or more audio characteristics of inputsub-signal 302 can be analyzed based on an analysis parameter. Likewise,for each window of windows 310, 320, and 330, one or more audiocharacteristics of input sub-signals 303 and 304 can also be analyzedbased on an analysis parameter. For each window of windows 310, 320, and330, input sub-signals 302, 303, and 304 can be prioritized based on theanalysis. Further, for each window of windows 310, 320, and 330, aninput sub-signal can be selected from input sub-signals 302, 303, and304 based on the prioritization, and the selected input sub-signal canbe converted into a haptic signal. Examples of analysis parameters caninclude: frequency, duration, envelope, density, and magnitude.

In one example, an analysis parameter can be a magnitude. In thisexample, for window 310, the magnitude of input sub-signals 302, 303,and 304 can be analyzed. Further, for window 310, input sub-signal 302can be selected because input sub-signal 302 has the highest magnitude.Input sub-signal 302 can subsequently be used to generate a hapticsignal for window 310. Likewise, for window 320, the magnitude of inputsub-signals 302, 303, and 304 can be analyzed. Further, for window 320,input sub-signal 302 can be selected because input sub-signal 302 hasthe highest magnitude. Input sub-signal 302 can subsequently be used togenerate a haptic signal for window 320. Similarly, for window 330, themagnitude of input sub-signals 302, 303, and 304 can be analyzed.Further, for window 330, input sub-signal 303 can be selected becauseinput sub-signal 303 has the highest magnitude. Input sub-signal 303 cansubsequently be used to generate a haptic signal for window 330.

In one example, an analysis parameter can be a density. In oneembodiment, a density can be a power, or energy, of a signal,distributed over the different frequencies of the signal. In thisexample, for window 310, the density of input sub-signals 302, 303, and304 can be analyzed. Further, for window 310, input sub-signal 304 canbe selected because input sub-signal 304 has the highest density. Inputsub-signal 304 can subsequently be used to generate a haptic signal forwindow 310. Likewise, for window 320, the density of input sub-signals302, 303, and 304 can be analyzed. Further, for window 320, inputsub-signal 304 can be selected because input sub-signal 304 has thehighest density. Input sub-signal 304 can subsequently be used togenerate a haptic signal for window 320. Similarly, for window 330, thedensity of input sub-signals 302, 303, and 304 can be analyzed. Further,for window 330, input sub-signal 303 can be selected because inputsub-signal 303 has the highest density. Input sub-signal 303 cansubsequently be used to generate a haptic signal for window 330.

FIG. 4 illustrates a flow diagram of haptic conversion functionalityperformed by a system, according to another embodiment of the invention.According to an embodiment, the haptic conversion functionality can beperformed by a software program or algorithm that receives a multimediafile, such as an audio file or video file, as input, and generates ahaptic signal as output. The software program or algorithm can be astand-alone software program, a mobile application, or a plug-in forother audio/video editing tools, such as ProTools. In anotherembodiment, the haptic conversion functionality can be performed by amultimedia player that receives a multimedia file and outputs thecontent of the multimedia file, where the content is augmented with oneor more haptic effects. The multimedia player can be adjusted for mobiledevices or computers. In one embodiment, the functionality may beperformed by a haptic conversion module (such as haptic conversionmodule 16 of FIG. 1).

According to the embodiment, the flow begins, and multimedia file 410 isreceived. Multimedia file 410 is any computer file that includesmultimedia data. In one example embodiment, multimedia file 410 is acomputer file that includes audio data. In another example embodiment,multimedia file 410 is a computer file that includes video data. Inanother example embodiment, multimedia file 410 is a computer file thatincludes both audio and video data. In yet another example embodiment,multimedia file 410 is a computer file that includes some other type ofdata. Multimedia file 410 can be streamed online or provided on aphysical digital support, such as a memory, disk, or other type ofnon-transitory computer-readable medium. Audio signal 420 (identified inFIG. 4 as audio track 420) is then extracted from multimedia file 410.Audio signal 420 is an example of an input signal that can be extractedfrom multimedia file 410. In alternate embodiments, audio signal 420 canbe replaced by another type of input signal, such as a video signal, anacceleration signal, an orientation signal, an ambient light signal, oranother type of signal that can include data captured with a sensor.

Audio signal 420 is subsequently filtered (e.g., band-pass filtered) tocreate a plurality of audio sub-signals, where an audio sub-signal is atype of input sub-signal (i.e., frequency band). More specifically, oneor more filters (e.g., band-pass filters) can be applied to audio signal420 to remove segments of audio signal 420, so that the remainingsegment of audio signal 420 includes one or more frequencies within aspecific frequency band. The segment of audio signal 420 that remainsafter the application of the filter is identified as an audiosub-signal. In embodiments involving a plurality of filters, each filtercan correspond to a specific frequency band, the plurality of filterscan be applied to audio signal 420 in multiple passes (where a differentfilter is applied to audio signal 420 in each pass), and each filter cancreate a segment of audio signal 420 that includes one or morefrequencies within a frequency band that corresponds to the filter. Inthe illustrated embodiments, the one or more filters are represented byband pass filters 430, 440, and 450. However, any number of filters canbe used. Further, any type of filter can be used. In certainembodiments, an audio sub-signal can include the entirety of audiosignal 420.

In certain embodiments, the choice of cutoff frequencies that aredefined for each filter can be done in a way to have contiguouscomplementary input sub-signals (i.e., frequency bands) that cover mostor all of the frequencies present in the original input signal. Thechoice of the number and bandwidths of the different filters can berelative to the nature of the input signal (e.g., audio signal 420), andcan affect the creation of a haptic signal. In some embodiments, theresonant frequency of a haptic output device can be used to define thenumber and bandwidths of the filters. For example, three filters can beused to produce three input sub-signals, where the first inputsub-signal includes content with frequencies lower than the resonantfrequency of the haptic output device, the second input sub-signalincludes content with frequencies around the resonant frequency of thehaptic output device, and the third input sub-signal includes contentwith frequencies greater than the resonant frequency of the hapticoutput device.

Next, audio sub-signals, or other types of input sub-signals, areconverted into haptic sub-signals using a plurality of haptic conversionalgorithms. In the illustrated embodiments, the haptic conversionalgorithms are represented by algorithms 431, 441, and 451. However, anynumber of haptic conversion algorithms can be used. In certainembodiments, each audio sub-signal is converted into a haptic signalusing a unique haptic conversion algorithm. Thus, the plurality ofhaptic conversion algorithms can covert the plurality of audiosub-signals, or other types of input sub-signals, into a plurality ofhaptic sub-signals.

Example haptic conversion algorithms can include: (a) multiplying theaudio sub-signal by a pre-determined factor and a sine carrier waveformexecuted at a haptic output device's resonant frequency; (b) multiplyingthe audio sub-signal by a pre-determined factor; or (c) shifting afrequency content of the audio sub-signal from a frequency band toanother frequency band that surrounds the haptic output device'sresonant frequency, and multiplying the shifted audio sub-signal by theoriginal audio sub-signal to preserve the shape of the original audiosub-signal. Examples of frequency shifting algorithms are furtherdescribed in conjunction with FIG. 5.

The haptic sub-signals are subsequently mixed into a haptic signal usinghaptic mixer 460. Haptic mixer 460 can mix the haptic sub-signalsaccording to one of a number of mixing techniques. Example mixingtechniques are further described in conjunction with FIG. 6.

In certain embodiments, the haptic signal can be normalized to 1 usingits maximum absolute value (not illustrated in FIG. 4). In otherembodiments, the normalizing can be omitted.

Further, in certain embodiments, one or more “noisy vibrations” can becleaned from the haptic signal (not illustrated in FIG. 4). A “noisyvibration” is a segment of a haptic signal that includes a deviationfrom a pre-defined value, where the deviation is below a pre-definedthreshold. This segment can be identified as “noise,” and can be removedfrom the haptic signal. This can cause the haptic signal to produce a“cleaner” (i.e., more adequate and compelling) haptic effect, when sentto a haptic output device.

Different techniques can be used to clean the haptic track from one ormore “noisy vibrations” that can be identified as “noise.” One techniquetakes a plurality of “chunks” or “windows” of samples from the hapticsignal, calculates an average absolute value (or a maximum absolutevalue in another implementation) of these samples, and identifies thesamples as “noise” if the calculated value for the sample is lower thana pre-defined threshold. All the samples identified as “noise” will thenhave their value reduced to 0. Another technique uses interleaving timewindows. For each time window, normalized values of the haptic signalare checked within two larger time windows: (a) the time window itselfand the preceding time window; and (b) the time window itself and thesucceeding time window. If in these two time windows, the averagenormalized absolute value (or the maximum normalized absolute value inanother implementation) is lower than a pre-defined threshold, thecontent in the time window is identified as “noise,” and its value isreduced to 0. In other embodiments, the cleaning of the one or morenoisy vibrations from the haptic signal can be omitted.

Subsequently, the haptic signal is sent to a device 470, where device470 is configured to generate and output one or more haptic effectsbased on the received haptic signal. Device 470 is further configured toreceive an input signal from multimedia file 410, where the input signalcan be an audio signal, a video signal, a signal that contains bothaudio data and video data, or some other type of input signal. Device470 can be further configured to generate and output one or moreeffects, such as audio effects, video effects, or other type of effects,based on the input signal. Further, device 470 can be further configuredto generate the one or more haptic effects so that they “complement” theaudio effects, video effects, or other type of effects. Device 470 canplay different haptic effects and/or signals simultaneously given itsconfiguration (i.e., number and position of haptic output devices).

FIG. 5 illustrates an example of shifting a frequency of an inputsignal, according to an embodiment of the invention. As previouslydescribes, a haptic conversion algorithm can convert an input signalinto a haptic signal by shifting a frequency content of the input signalfrom a frequency band to another frequency band that surrounds a hapticoutput device's resonant frequency, and by eventually multiplying theshifted input signal by the original input signal to preserve the shapeof the original input signal. In the example illustrated in FIG. 5, aninput signal 510 is frequency-shifted into a haptic signal 520. Morespecifically, input signal 510 includes frequency content within afrequency band of 300 Hz to 500 Hz. However, when input signal 510 isfrequency-shifted into haptic signal 520, the frequency content of inputsignal is shifted from the frequency band of 300 Hz to 500 Hz to afrequency band of 100 Hz to 200 Hz. Thus, haptic signal 520 includesfrequency content within the frequency band of 100 Hz to 200 Hz.

In certain embodiments, the frequency-shifting is achieved using a fastFourier transform of input signal 510. Further, one of multiplefrequency-shifting techniques can be used, depending on the sizes of a“shift-from” frequency band (e.g., an original frequency band of inputsignal 510) and a “shift-to” frequency band (e.g., a shifted frequencyband of haptic signal 520). This is because, depending on the sizes ofthe shift-from frequency band and the shift-to frequency band, eithermultiple frequencies within the shift-from frequency band are replacedwith a single frequency within the shift-to frequency band, or a singleband within the shift-from frequency band is replaced with multiplefrequencies within the shift-to frequency band.

In scenarios where a shift-from frequency band is larger than a shift-tofrequency band, each frequency of the shift-to frequency band representsmultiple frequencies of the shift-from frequency band. To accomplishthis, one of three frequency-shifting techniques can be used, accordingto certain embodiments. The first frequency-shifting technique is toaverage the fast Fourier transform values of the multiple frequencies ofthe shift-from frequency band, and to assign the average to thecorresponding frequency of the shift-to frequency band. The secondfrequency-shifting technique is to sum the fast Fourier transform valuesof the multiple frequencies of the shift-from frequency band, and toassign the sum to the corresponding frequency of the shift-to frequencyband. The third frequency-shifting technique is to select a frequency ofthe shift-from frequency band that has the largest absolute fast Fouriertransform value, ignore the other frequencies of the shift-fromfrequency band, and to assign the largest absolute fast Fouriertransform value to the corresponding frequency of the shift-to frequencyband.

On the other hand, in scenarios where a shift-to frequency band islarger than a shift-from frequency band, each frequency of theshift-from frequency band is represented by multiple frequencies of theshift-to frequency band. To accomplish this, one of twofrequency-shifting techniques can be used. The first frequency-shiftingtechnique is to assign the fast Fourier transform value of the frequencyof the shift-from frequency band to the multiple frequencies of theshift-to frequency band. The second frequency-shifting technique is toassign the fast Fourier transform value of the frequency of theshift-from frequency band to a single frequency (e.g., a lowestfrequency) of the shift-to frequency band.

FIG. 6 illustrates a block diagram of a haptic mixer 610 configured tomix a plurality of haptic sub-signals, according to an embodiment of theinvention. As previously described, an input signal, such as audiosignal 620, can be converted into a plurality of haptic sub-signals(illustrated in FIG. 6 as haptic tracks), where haptic mixer 610 can mixthe haptic sub-signals into a haptic signal. Haptic mixer 610 can mixthe haptic sub-signals according to one of a number of mixingtechniques, where three example mixing techniques are illustrated inFIG. 6 at 630.

According to the first mixing technique, at 631, the haptic sub-signalsare summed, and the sum of the haptic sub-signals is used to calculate ahaptic signal. In certain embodiments, the haptic signal is subsequentlynormalized.

According to the second mixing technique, at 632, each haptic sub-signalis segmented into a plurality of time windows. At 633, for each timewindow, one or more dominant frequencies in the corresponding inputsignal are identified. For each time window, a power spectrum density(“PSD”) value is calculated for each frequency in the original inputsignal and N frequencies having the highest PSD values are identified asbeing the “dominant frequencies,” where N is any number of frequencies.These N frequencies belong to the different bands represented by thedifferent input sub-signals. At 634, the band having the highest numberof frequencies in the group of the N dominant frequencies is identifiedas the dominant band. Its corresponding haptic sub-signal values areassigned to the resulting output haptic signal at the specific timewindow.

According to the third mixing technique, at 635, each input sub-signalis segmented into a plurality of time windows. At 636, for each timewindow, a PSD value is calculated per frequency band, and a PSDpercentage contribution is also calculated per frequency band for eachinput sub-signal. More specifically, for each time window, a PSD valuefor each input sub-signal is calculated, an overall PSD value for thetime window calculated, and a ratio of an input sub-signal PSD value tothe overall PSD value is calculated for each input sub-signal. The ratioof the input sub-signal PSD value to the overall PSD value is the PSDpercentage contribution for that specific input sub-signal. At 637, foreach time window, each haptic sub-signal is weighted according to itscorresponding input sub-signal PSD percentage contribution, the weightedhaptic sub-signals are summed, and the haptic signal is calculated basedon the weighted sum of haptic sub-signals of each time window.

The third mixing technique stems from the fact that the frequencycontent can change significantly throughout an input sub-signal, andthus, can change significantly through a haptic sub-signal. When hapticsub-signals are added together, each haptic sub-signal is given equalweight. However, this may not take into consideration such situations aswhen an original input sub-signal had a slight influence on the originalinput content. By giving a weight related to a PSD percentagecontribution, each haptic sub-signal will contribute to the hapticsignal similar to its corresponding input sub-signal's contribution tothe original input signal. The resulting haptic signal for a specifictime window is thus the sum of all the haptic sub-signals, where thehaptic sub-signals are each weighted by the contribution of itscorresponding input sub-signal to the original input signal for thespecific time window.

FIG. 7 illustrates a flow diagram of the functionality of a hapticconversion module (such as haptic conversion module 16 of FIG. 1),according to an embodiment of the invention. The flow begins andproceeds to 710. At 710, an input is received. In certain embodiments, asegment of an input signal can be received. In other embodiments, amultimedia file can be received, and an input signal can be extractedfrom the multimedia file. In certain embodiments, the input signal canbe an audio signal. In other embodiments, the input signal can be avideo signal. In other embodiments, the input signal can be anacceleration signal. The flow then proceeds to 720.

At 720, the input is segmented into a plurality of input sub-signals. Incertain embodiments, the input can be filtered using one or morefilters, where each input sub-signal can include a frequency band.Further, in some of these embodiments, the one or more filters includeat least one band-pass filter. The flow then proceeds to 730.

At 730, the input sub-signals are converted into a haptic signal. Incertain embodiments, the input sub-signals can be prioritized based onan analysis parameter. One or more input sub-signals can be selected,and the haptic signal can be generated based on the selected inputsub-signals. In some of these embodiments, the analysis parameter caninclude a characteristic of the input sub-signals. Further, thecharacteristic of the input sub-signals can include one of: a frequency,a duration, an envelope, a density, or a magnitude. Even further, insome of these embodiments, the haptic signal can be warped into a warpedhaptic signal, where one or more haptic effects can be generated basedon the warped haptic signal.

In other embodiments, the input sub-signals can be converted into hapticsub-signals. In some of these embodiments, the input sub-signals can beconverted into haptic sub-signals by at least one of: multiplying aninput sub-signal by a factor and a sine carrier waveform; multiplyingthe input sub-signal by a factor; or shifting frequency content from afirst frequency band of the input sub-signal to a second frequency bandof the input sub-signal. Further, in some of these embodiments, eachinput sub-signal can be converted into a haptic sub-signal using aunique haptic conversion algorithm. Even further, in some embodiments, afast Fourier transform of the input sub-signal can be performed.

Furthermore, in embodiments where frequency content is shifted from afirst frequency band of the input sub-signal to a second frequency bandof the input sub-signal, the shifting can include at least one of:averaging a plurality of fast Fourier transform values for a pluralityof frequencies within the first frequency band of the input sub-signal,and assigning the average to a frequency within the second frequencyband of the input sub-signal; summing a plurality of fast Fouriertransform values for a plurality of frequencies within the firstfrequency band of the input sub-signal, and assigning the sum to afrequency within the second frequency band of the input sub-signal;selecting a fast Fourier transform value that has a highest absolutevalue from a plurality of fast Fourier transform values for a pluralityof frequencies within the first frequency band of the input sub-signal,and assigning the selected fast Fourier transform value to a frequencywithin the second frequency band of the input sub-signal; assigning afast Fourier transform value for a frequency within the first frequencyband of the input sub-signal to a plurality of frequencies within thesecond frequency band of the input sub-signal; or assigning a fastFourier transform value for a frequency within the first frequency bandof the input sub-signal to a lowest frequency within the secondfrequency band of the input sub-signal.

In certain embodiments, the haptic sub-signals can then be mixed intothe haptic signal. In some of these embodiments, the mixing can includeone of: summing the plurality of haptic sub-signals into the hapticsignal and normalizing the haptic signal; segmenting the input signalinto one or more time-windows, analyzing a plurality of frequencies ofthe plurality of input sub-signals for each time-window, and selecting ahaptic sub-signal from the plurality of haptic sub-signals as the hapticsignal for each time-window, where the selected haptic sub-signalincludes a frequency of the plurality of frequencies; or segmenting thehaptic signal into one or more time windows, calculating a powerspectrum density percentage contribution for each input sub-signal ofthe plurality of input sub-signals for each time window, and calculatinga weighted combination of the plurality of haptic sub-signals as thehaptic signal for each time-window, where a weight of each hapticsub-signal is based on the power spectrum density percentagecontribution of the corresponding input sub-signal.

In certain embodiments, the haptic signal can be normalized to 1 usingits maximum absolute value. Further, in certain embodiments, one or morenoisy vibrations can be cleaned from the haptic signal. In some of theseembodiments, one or more sample haptic sub-signals can be selected fromthe haptic signal. A mean absolute value can be calculated for eachselected sample haptic sub-signal of the one or more selected samplehaptic sub-signals. Each mean absolute value can be compared with athreshold. A sample haptic sub-signal can be removed from the hapticsignal when its corresponding mean absolute value is less than thethreshold. The flow then proceeds to 740.

At 740, one or more haptic effects are generated based on the hapticsignal. In some embodiments, the haptic signal can be sent to a hapticoutput device to generate the one or more haptic effects. In some ofthese embodiments, the haptic output device can be an actuator. The flowthen ends.

Thus, in one embodiment, a system can filter an input into one or morefrequency bands, analyze and prioritize the one or more frequency bandsbased on one or more pre-determined analysis parameters, select at leastone of the frequency bands based on the prioritization, and can use theselected frequency band(s) to generate a haptic signal that isultimately used to generate one or more haptic effects. By analyzingmultiple frequency bands of an input and selecting one or more frequencybands based on a prioritization, an opportunity is given for the entireinput to come through in the haptic signal. More specifically, an entirefrequency spectrum of the input can be encompassed, and one or morespecific frequency bands can be selected, such as one or more frequencybands that are mostly in the foreground from the perspective of theuser. This can lead to a haptic effect that is more “customized” to theinput, rather than merely selecting a frequency band of the inputwithout first analyzing the multiple frequency bands of the input. Byfiltering the input and prioritizing the multiple frequency bands of theinput, the system can further “perfect” the haptic effect that can begenerated to “complement” the input. Further, such a solution can be anelegant solution that can be extended by future haptic conversionalgorithms.

Further, in another embodiment, a system can filter an input into one ormore frequency bands, convert each frequency band into a hapticsub-signal, and mix the haptic sub-signals into a haptic signal that isultimate used to generate one or more haptic effects. The system cancreate a compelling haptic effect that “complements” the input withoutthe need of any human intervention, such as authored effects. Such asystem can be implemented on any mobile device, such as a tablet orsmartphone, and can use the processing power of the device as well as ahaptic playback component of the device to deliver a richer experience,such as a richer video viewing experience or richer music listeningexperience). Further, unlike previous solutions, the system can attemptto create a haptic signal based on all of the input, rather than aspecific segment of the input, such as a low frequency content of theinput. As previously described, this can be accomplished by regroupingthe content of the input given the existing frequencies, and processingeach group adequately according to the group's frequency profile, andaccording to the haptic play device. Such an approach can produce a more“perfect” haptic effect that “complements” the input.

The features, structures, or characteristics of the invention describedthroughout this specification may be combined in any suitable manner inone or more embodiments. For example, the usage of “one embodiment,”“some embodiments,” “certain embodiment,” “certain embodiments,” orother similar language, throughout this specification refers to the factthat a particular feature, structure, or characteristic described inconnection with the embodiment may be included in at least oneembodiment of the present invention. Thus, appearances of the phrases“one embodiment,” “some embodiments,” “a certain embodiment,” “certainembodiments,” or other similar language, throughout this specificationdo not necessarily all refer to the same group of embodiments, and thedescribed features, structures, or characteristics may be combined inany suitable manner in one or more embodiments.

One having ordinary skill in the art will readily understand that theinvention as discussed above may be practiced with steps in a differentorder, and/or with elements in configurations which are different thanthose which are disclosed. Therefore, although the invention has beendescribed based upon these preferred embodiments, it would be apparentto those of skill in the art that certain modifications, variations, andalternative constructions would be apparent, while remaining within thespirit and scope of the invention. In order to determine the metes andbounds of the invention, therefore, reference should be made to theappended claims.

We claim:
 1. A non-transitory computer-readable medium havinginstructions stored thereon that, when executed by a processor, causethe processor to convert an input into one or more haptic effects usingsegmenting and combining, the converting comprising: receiving theinput; segmenting the input into input sub-signals, the segmentingcomprising filtering the input using one or more filters, wherein eachof the input sub-signals comprises a different frequency band that isoutput from a respective filter; after the segmenting, prioritizing theinput sub-signals based on an analysis parameter, wherein theprioritizing comprises determining for each of the input sub-signals, avalue of a characteristic that is defined by the analysis parameter ofthe input sub-signal, and ordering the input sub-signals based on thedetermined values; selecting a subset of the input sub-signals from theinput sub-signals based on the prioritizing, wherein the subset is lessthan the input sub-signals; calculating a haptic signal based on acombination of the selected subset of input sub-signals, the calculatingthe haptic signal including: converting the subset of input sub-signalsinto haptic sub-signals wherein the converting includes multiplying aninput sub-signal by a factor, and summing the haptic sub-signals intothe haptic signal and normalizing the haptic signal; and generating theone or more haptic effects based on the haptic signal.
 2. Thecomputer-readable medium of claim 1, wherein the one or more filterscomprise at least one band-pass filter.
 3. The computer-readable mediumof claim 1, wherein the receiving the input further comprises receivinga segment of an input signal.
 4. The computer-readable medium of claim3, wherein the analysis parameter comprises a maximum magnitude valuefor each of the input sub-signals.
 5. The computer-readable medium ofclaim 1, wherein the characteristic comprises one of: a frequency, aduration, an envelope, a density, or a magnitude.
 6. Thecomputer-readable medium of claim 1, the converting further comprising:warping the haptic signal into a warped haptic signal, the warping basedon a type of haptic output device that is configured to play the warpedhaptic signal; wherein the generating the one or more haptic effectsfurther comprises generating the one or more haptic effects based on thewarped haptic signal.
 7. The computer-readable medium of claim 1,wherein the receiving the input further comprises: receiving amultimedia file; and extracting an input signal from the multimediafile.
 8. The computer-readable medium of claim 1, wherein the convertingthe subset of input sub-signals into haptic sub-signals furthercomprises at least one of: multiplying an input sub-signal by a factorand a sine carrier waveform; multiplying the input sub-signal by afactor; or shifting frequency content from a first frequency band of theinput sub-signal to a second frequency band of the input sub-signal. 9.The computer-readable medium of claim 8 wherein each of the inputsub-signals is converted into a haptic sub-signal using a unique hapticconversion algorithm.
 10. The computer-readable medium of claim 8,wherein the shifting the frequency content from the first frequency bandof the input sub-signal to the second frequency band of the inputsub-signal further comprises performing a fast Fourier transform of theinput sub-signal.
 11. The computer-readable medium of claim 10, whereinthe shifting the frequency content from the first frequency band of theinput sub-signal to the second frequency band of the input sub-signalfurther comprises at least one of: averaging fast Fourier transformvalues for frequencies within the first frequency band of the inputsub-signal, and assigning the average to a frequency within the secondfrequency band of the input sub-signal; summing fast Fourier transformvalues for frequencies within the first frequency band of the inputsub-signal, and assigning the sum to a frequency within the secondfrequency band of the input sub-signal; selecting a fast Fouriertransform value that has a highest absolute value from fast Fouriertransform values for frequencies within the first frequency band of theinput sub-signal, and assigning the selected fast Fourier transformvalue to a frequency within the second frequency band of the inputsub-signal; assigning a fast Fourier transform value for a frequencywithin the first frequency band of the input sub-signal to frequencieswithin the second frequency band of the input sub-signal; or assigning afast Fourier transform value for a frequency within the first frequencyband of the input sub-signal to a lowest frequency within the secondfrequency band of the input sub-signal.
 12. The computer-readable mediumof claim 1, wherein the summing of the haptic sub-signals into thehaptic signal comprises at least one of: segmenting the haptic signalinto one or more time-windows, analyzing frequencies of the hapticsub-signals for each time-window, and selecting a haptic sub-signal fromthe haptic sub-signals as the haptic signal for each time-window,wherein the selected haptic sub-signal comprises a frequency of thefrequencies; or segmenting the input signal into one or more timewindows, calculating a power spectrum density percentage contributionfor each of the input sub-signals of the input sub-signals for each timewindow, and calculating a weighted combination of the correspondinghaptic sub-signals as the haptic signal for each time-window, wherein aweight of each haptic sub-signal is based on the power spectrum densitypercentage contribution of each corresponding input sub-signal.
 13. Thecomputer-readable medium of claim 1, the converting further comprising:normalizing the haptic signal to 1 using its maximum absolute value. 14.The computer-readable medium of claim 1, the converting furthercomprising: cleaning one or more noisy vibrations from the hapticsignal.
 15. The computer-readable medium of claim 14, wherein thecleaning the one or more noisy vibrations from the haptic signal furthercomprises: selecting one or more sample haptic sub-signals from thehaptic signal; calculating a mean absolute value for each selectedsample haptic sub-signal of the one or more selected sample hapticsub-signals; comparing each mean absolute value with a threshold; andremoving a sample haptic sub-signal from the haptic signal when itscorresponding mean absolute value is less than the threshold.
 16. Thecomputer-readable medium of claim 1, wherein the generating the one ormore haptic effects further comprising sending the haptic signal to ahaptic output device to generate the one or more haptic effects.
 17. Thecomputer-readable medium of claim 16, wherein the haptic output devicecomprises an actuator.
 18. The computer-readable medium of claim 1,wherein the input comprises one of an audio signal, a video signal, oran acceleration signal.
 19. A computer-implemented method for convertingan input into one or more haptic effects using segmenting and combining,the computer-implemented method comprising: receiving the input;segmenting by a processor the input into input sub-signals, thesegmenting comprising filtering the input using one or more filters,wherein each of the input sub-signals comprises a different frequencyband that is output from a respective filter; after the segmenting,prioritizing by the processor the input sub-signals based on an analysisparameter, wherein the prioritizing comprises determining for each ofthe input sub-signals, a value of a characteristic that is defined bythe analysis parameter of the input sub-signal, and ordering the inputsub-signals based on the determined values; selecting by the processor asubset of the input sub-signals from the input sub-signals based on theprioritizing, wherein the subset is less than the input sub-signals;calculating a haptic signal based on a combination of the selectedsubset of input sub-signals, the calculating the haptic signalincluding: converting the subset of input sub-signals into hapticsub-signals wherein the converting includes multiplying an inputsub-signal by a factor, and summing the haptic sub-signals into thehaptic signal and normalizing the haptic signal; and generating the oneor more haptic effects based on the haptic signal.
 20. Thecomputer-implemented method of claim 19, wherein the analysis parametercomprises a maximum magnitude value for each of the input sub-signals.21. The computer-implemented method of claim 19, wherein the receivingthe input further comprises: receiving a multimedia file; and extractingan input signal from the multimedia file.
 22. A system for converting aninput into one or more haptic effects using segmenting and combining,the system comprising: a memory configured to store a haptic conversionmodule; and a processor configured to execute the haptic conversionmodule stored on the memory; wherein the haptic conversion module isconfigured to receive the input; wherein the haptic conversion module isfurther configured to segment the input into input sub-signals, thesegmenting comprising filtering the input using one or more filters,wherein each of the input sub-signals comprises a different frequencyband that is output from a respective filter; wherein the hapticconversion module is further configured to, after the segmenting,prioritize the input sub-signals based on an analysis parameter, whereinthe prioritize comprises determining for each of the input sub-signals,a value of a characteristic that is defined by the analysis parameter ofthe input sub-signal, and ordering the input sub-signals based on thedetermined values; wherein the haptic conversion module is furtherconfigured to select a subset of the input sub-signals from the inputsub-signals based on the prioritizing, wherein the subset is less thanthe input sub-signals; wherein the haptic conversion module is furtherconfigured to calculate a haptic signal based on a combination of theselected subset of input sub-signals, the calculating the haptic signalincluding: converting the subset of input sub-signals into hapticsub-signals wherein the converting includes multiplying an inputsub-signal by a factor, and summing the haptic sub-signals into thehaptic signal and normalizing the haptic signal; and wherein the hapticconversion module is further configured to generate the one or morehaptic effects based on the haptic signal.
 23. The system of claim 22,wherein the analysis parameter comprises a maximum magnitude value foreach of the input sub-signals.
 24. The system of claim 22, wherein thehaptic conversion module is further configured to receive a multimediafile; and wherein the haptic conversion module is further configured toextract an input signal from the multimedia file.
 25. Thecomputer-implemented method of claim 19, wherein the generating the oneor more haptic effects further comprises sending the haptic signal to ahaptic output device.
 26. The system of claim 22, wherein the one ormore haptic effects is generated by sending the haptic signal to ahaptic output device.
 27. The computer-implemented method of claim 21,wherein the mixing the haptic sub-signals into the haptic signalcomprises at least one of: segmenting the haptic signal into one or moretime-windows, analyzing frequencies of the haptic sub-signals for eachtime-window, and selecting a haptic sub-signal from the hapticsub-signals as the haptic signal for each time-window, wherein theselected haptic sub-signal comprises a frequency of the frequencies; orsegmenting the input signal into one or more time windows, calculating apower spectrum density percentage contribution for each of the inputsub-signals of the input sub-signals for each time window, andcalculating a weighted combination of the corresponding hapticsub-signals as the haptic signal for each time-window, wherein a weightof each haptic sub-signal is based on the power spectrum densitypercentage contribution of each corresponding input sub-signal.